Characterization of Cued Speech Vowels from The inner LiP contour.

4 pagesInternational audienceCued Speech (CS) is a manual code that complements lip-reading to enhance speech perception from visual input. The phonetic translation of CS gestures needs to combine the manual CS information with information from the lips, taking into account the desynchronization delay (Attina et al. [1], Aboutabit et al. [2]) between these two flows of information. This paper focuses on the analysis of the lip flow for vowels in French Cued Speech. The vocalic lip targets are defined automatically at the instant of minimum velocity of the inner lip contour area parameter, constrained by the corresponding acoustic labeling. We discuss in particular the possibility of discriminating the vowels with geometric lip parameters using the values at the instant of vocalic targets when associated to a Cued Speech hand position

Cued Speech Automatic Recognition in Normal Hearing and Deaf Subjects

International audienceThis article discusses the automatic recognition of Cued Speech in French based on hidden Markov models (HMMs)

Adaptation de la production labiale d'un participant sourd et classification : le cas des voyelles en contexte du code LPC.

International audienceThe phonetic translation of Cued Speech (CS) gestures needs to mix the manual CS information together with the lips, taking into account the desynchronization delay (Attina et al. [2], Aboutabit et al. [7]) between these two flows of information. This contribution focuses on the lip flow modeling in the case of French vowels. Previously, classification models have been developed for a professional normal hearing CS speaker (Aboutabit et al., [7]). These models are used as a reference. Now, we process the case of a deaf CS speaker and discuss the possibilities of classification. The best performance (92,8%) is obtained with the adaptation of the deaf data to the reference models.Dans un système de communication entre des personnes normo entendantes et des personnes malentendantes, la transcription phonétique du code LPC nécessite de fusionner l'information issue des gestes de main et de lèvres. Cette contribution est centrée sur le traitement du flux labial dans le cas des voyelles. Des modèles de classification ont été développé pour un participant normo-entendant pratiquant le LPC. (Aboutabit et al., [7]). Ces modèles sont utilisés dans cette contribution comme une référence pour étudier les possibilités de classification des voyelles produites par un codeur LPC sourd. La meilleure performance (92,8%) est obtenue avec une adaptation des données "sourd" au modèles de référence

Reconnaissance de la Langue Française Parlée Complété (LPC) : décodage phonétique des gestes main-lèvres.

Cued Speech (CS) is a visual communication system that uses handshapes placed in different positions near the face, in combination with the natural speech lip-reading, to enhance speech perception from visual input for deaf people. In this system, the speaker moves his hand in close relation with speech. Handshapes are designed to distinguish among consonants whereas hand positions are used to distinguish among vowels. Due to the CS system, both manual and lip flows produced by the CS speaker carry a part of the phonetic information. This work presents at first a method for the automatic coding of the manual flow in term of CS hand positions and CS handshapes. Then the lip-shape classification of the vowels and the consonants is discussed. The labial flow is composed of the temporal variations of lip parameters extracted from the inner and the outer contours of the lips. This work will show how the distribution of lip parameters inside each group of CS hand positions allows vowel discrimination. A classification method based on Gaussian modeling is presented and results demonstrate a good performance of this classification (89% as test score). The vocalic context is taken into account in the case of the consonants, with the use of HMM for the modeling of the lip transition from the consonant towards the vowel (80 % as test scores in term of CV visemes). Finally, the modeling of the lip information and the coding of the manual flow are included in a “Master-Slave” fusion model for recognition of the vowels and the consonants in the CS context. The fusion model integrates the temporal constraints of the CS production and perception. This work is thus also a first contribution to the modeling of the CS system from the perceptive point of view.La Langue Française Parlée Complétée (LPC) héritée du Cued Speech (CS) a été conçue pour compléter la lecture labiale par nature ambigüe et ainsi améliorer la perception de la parole par les sourds profonds. Dans ce système, le locuteur pointe des positions précises sur le côté de son visage ou à la base du cou en présentant de dos des formes de main bien définies. La main et les lèvres portent chacune une partie complémentaire de l'information phonétique. Cette thèse présente tout d'abord une modélisation du flux manuel pour le codage automatique des positions de la main et de la configuration. Puis les travaux sont centrés sur le flux labial en discutant la classification des voyelles et des consonnes du Français. Le flux labial est composé des variations temporelles de paramètres caractéristiques issus du contour interne et externe des lèvres. Dans le cas des voyelles la méthode de classification utilise la modélisation gaussienne et les résultats montrent une performance moyenne de 89 % en fonction de la position de la main LPC. Le contexte vocalique est pris en compte dans le cas des consonnes par une modélisation HMM de la transition labiale de la consonne vers la voyelle avec un taux d'identification de 80 % en termes de visèmes CV. Un modèle de fusion « Maître-Esclave » piloté par le flux manuel est présenté et discuté dans le cadre de la reconnaissance des voyelles et des consonnes produites en contexte LPC. Le modèle de fusion prend en compte les contraintes temporelles de la production et la perception du LPC, ce qui constitue aussi une première contribution à la modélisation du système LPC du point de vue perceptif

Reconnaissance de la Langue Française Parlée Complété (LPC) : décodage phonétique des gestes main-lèvres

GRENOBLE1-BU Sciences (384212103) / SudocGRENOBLE-GIPSA-lab (384212301) / SudocSudocFranceF

Alternative speech communication based on cued speech

International audienceThis study focuses on alternative speech communication based on Cued Speech. Cued Speech is a visual mode of communication that uses hand shapes and placements in combination with the mouth movements of speech to make the phonemes of a spoken language look different from each other and clearly understandable to deaf and hearingimpaired people. Originally, the aim of Cued Speech was to overcome the problems of lip reading and thus enable deaf children and adults to wholly understand spoken language. In this study, however, we investigate the use of Cued Speech not only for perception, but also for speech production in the case of speech- or hearing-impaired individuals. The proposed method is based on hidden Markov model (HMM) automatic recognition. Automatic recognition of Cued Speech and conversion to text, audio, or synthesized Cued Speech can be served as an alternative speech communication method for individuals with speech or hearing impairments. This article presents vowel and consonant, and also isolated word recognition experiments for Cued Speech for French. The results obtained are promising and comparable to the results obtained when using audio signal

Alternative speech communication based on cued speech

International audienceThis study focuses on alternative speech communication based on Cued Speech. Cued Speech is a visual mode of communication that uses hand shapes and placements in combination with the mouth movements of speech to make the phonemes of a spoken language look different from each other and clearly understandable to deaf and hearingimpaired people. Originally, the aim of Cued Speech was to overcome the problems of lip reading and thus enable deaf children and adults to wholly understand spoken language. In this study, however, we investigate the use of Cued Speech not only for perception, but also for speech production in the case of speech- or hearing-impaired individuals. The proposed method is based on hidden Markov model (HMM) automatic recognition. Automatic recognition of Cued Speech and conversion to text, audio, or synthesized Cued Speech can be served as an alternative speech communication method for individuals with speech or hearing impairments. This article presents vowel and consonant, and also isolated word recognition experiments for Cued Speech for French. The results obtained are promising and comparable to the results obtained when using audio signal

Hand and Lip desynchronization analysis in French Cued Speech: Automatic temporal segmentation of hand flow

4International audienceIn the context of Cued Speech gesture phonetic translation, the automatic recognition of lip and hand movements is a key factor. The hand and the lip parameters are not synchronized, thus the fusion of the two channels (hand and lips) needs the knowledge of the desynchronized delay. This contribution focuses on the presentation of an automatic algorithm for temporal segmentation of the hand cue information based on Gaussian modeling of the hand position and minimum of velocity. The segmentation delivers the beginning of the hand transition and the instant of attained position. The hand segmentation is used to calculate the delay between hand and lip targets, in relation with the corresponding acoustic realization in the case of French CV syllables extracted from a corpus of phrases uttered and coded by a Cued Speech speaker. This study confirms in a more complex context the importance of the instant of attained hand position as pointed out by Attina and colleagues, in terms of control and for the fusion process

Alternative speech communication based on cued speech

International audienceThis study focuses on alternative speech communication based on Cued Speech. Cued Speech is a visual mode of communication that uses hand shapes and placements in combination with the mouth movements of speech to make the phonemes of a spoken language look different from each other and clearly understandable to deaf and hearingimpaired people. Originally, the aim of Cued Speech was to overcome the problems of lip reading and thus enable deaf children and adults to wholly understand spoken language. In this study, however, we investigate the use of Cued Speech not only for perception, but also for speech production in the case of speech- or hearing-impaired individuals. The proposed method is based on hidden Markov model (HMM) automatic recognition. Automatic recognition of Cued Speech and conversion to text, audio, or synthesized Cued Speech can be served as an alternative speech communication method for individuals with speech or hearing impairments. This article presents vowel and consonant, and also isolated word recognition experiments for Cued Speech for French. The results obtained are promising and comparable to the results obtained when using audio signal